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C.  H.  CHEN 


I.  INTRODUCTION 

'  i 

■  ^Gaussian  assumption  has  been  a  fundamental  one  In  most  statistical  signed 
processing  work.  The  assunption  not  only  simplifies  the  analytical  problems 
involved  but  also  matches  the  data  characteristics  in  many  cases  because  of 
the  law  of  large  numbers.  In  a  number  of  Navy  scnar,  radar  and  comnunicaticns 
systems,  signal  processing  algorithms  must  be  developed  without  the  Gaussian 
assunption .  The  need  for  ncn-Gaussian  signal  processing 

study  was  well  described  in  a  recent  CNR  SRO  booklet.  In  our  view  the  follow¬ 
ing  is  a  partial  list  of  a  broad  spectrum  of  correlated  problem  areas  in  ncn- 
Gaussian  signal  processing. 


1.  To  investigate  non- linear  adaptive  procedures  to  assess  their 
detection  performance  against  transient  signals. 

2.  To  investigate  non- linear  spectral  analysis  techniques  such  as 
the  Maximum  Entrqpy  Spectral  Analysis  (MESA)  and  the  Maximum 
Likelihood  Method  (MLM)  to  assess  their  detection  performance 
against  transient  signals. 

3.  To  investigate  techniques  for  weak  signal  extraction  in  the  pre¬ 
sence  of  strong  interf erring  signals.  As  a  special  case,  potential 
techniques  for  detecting  broad-band  (such  as  spread-spectrum)  signals 
can  be  explored  including  the  analysis  of  changes  in  noise  statistics 
caused  by  pseudo-noise  signals. 

4.  To  explore  the  use  of  ncnparametric  and  robust  statistical  approaches 


5.  To  study  ncn-Gaussian  models  appropriate  for  reverberation  or 
reverberation-like  noise  found  in  shallow  water  environment. 

€.  To  examine  the  effects  of  ncn-Gaussian  sources  of  irrpulsive  noise 
interference  in  tine-delay  estimation,  passive  range  estimation  and 
related  underwater  acoustic  signal  processing  problems. 

7.  TO  enlarge  the  basic  mathematical  and  statistical  theory  of  ncn- 
Gaussian  stochastic  processes  applicable  to  both  electromagnetic 
and  underwater  sound  environments  . 

8.  To  characterise  the  ncn-Gaussian  channels  and  interferences  including 
the  multipath,  dispersion,  doppler  effects  and  the  optimal  reception 
under  these  circims  tances . 

9.  To  examine  the  modifications  required  in  the  existing  signal  processing 
system  hardware  for  ncn-Gaussian  signal  processing. 

Obviously  good  solutions  to  the  above  problems  require  expertise  in  several 
areas  such  as  cxmnunications  and  information  theory,  pattern  recognition,  wave 
propagation,  digital  signal  processing,  image  processing,  and  mathematical  sta¬ 
tistics,  etc.  As  the  problems  are  interrelated,  they  should  be  treated  jointly 
as  much  as  possible.  Problems  1  and  2  are  examined  in  detail  in  this  report. 

For  Problem  3,  detection  performance  of  spread-spectrum  signals  and  the  extraction 
of  weak  signals  in  Gaussian  noise  have  been  considered  even  though  further  work,  is 
much  needed.  For  a  general  discussion  of  spread-spectrun  systems,  see  [1], 
Adaptive  digital  filtering  and  Kalman  filtering  are  suitable  techniques  for 
extraction  of  broad-band  non-Gaussian  signals  in  noises.  However  analytical 
work  on  the  detection  performance  is  not  available.  For  Problem  4,  same  exist¬ 
ing  analytical  work  cn  robust  and  ncnparametric  detection  of  signals  in  impulsive 


t 


-  3  - 


noise  and  randan  interference  (See  [2]  and  the  references  listed  in  the  paper) 
are  useful.  While  general  theoretical  work  based  cn  general  assumptions  is 
very  important,  special  cases  should  be  considered  so  that  the  detection  pro¬ 
cedures  and  performances  developed  will  match  more  closely  to  specific  impul¬ 
sive  noise  and  randan  interference  conditions  considered.  Proper  mathematical 
modelling  of  impulsive  noise,  randan  interference  and  channels  is  thus  required, 
which  is  the  Problem  5  stated  above. 

Time-delay  estimation  and  passive  ranging  estimation  from  an  array  of 
sensors  are  typical  scalar  signal  processing  problems  (see  e.g.  [3]  [4]).  Pro¬ 
blem  6  suggests  that  the  non-Gaussian  sources  be  taken  into  consideration  as 
the  existing  work  based  cn  Gaussian  assunpticn  is  quite  restrictive.  Problem  7 
presents  a  nunber  of  mathematical  and  statistical  topics  to  be  explored.  In 
fact  sene  theoretical  work  for  Gaussian  signals  and  Gaussian  noises  can  be  ex¬ 
tended  to  non-Gaussian  cases  [5].  For  example,  consider  the  detection  and  esti¬ 
mation  of  Gaussian  signal  in  Gaussian  noise.  The  conditional  mean  of  the  signal 
is  the  best  estimate  for  a  variety  of  criteria.  In  the  non-Gaussian  case  it  is 
the  best  linear  estimate  in  the  sense  that  mean-squared  error  is  minimized.  When 
the  signal  is  unknown  but  non-randan,  the  maximum  likelihood  estimate  of  the 
signal  is  the  same  whether  the  noise  is  Gaussian  or  non-Gaussian.  The  test  sta¬ 
tistics  for  detecting  ncn-Gaussian  signal  in  Gaussian  noise  is  the  same  as  that 
for  detecting  Gaussian  signal  in  Gaussian  noise  at  small  signal  condition.  Many 
other  mathematical  properties  remain  to  be  examined. 

Problem  8  has  been  well  explored  and  reported  in  publications  in  ccnmuni ca¬ 
tions,  wave  propagation  and  other  areas.  However,  much  less  amount  of  work  has 
been  dene  for  underwater  transmission  media.  Although  most  signed  processing 
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hardware  designed  for  Gaussian  signal  processing  can  also  be  used  far  ncn- 
Gaussian  signed  processing  with  ncn-op timed  performance,  a  modification  of 
the  hardware  is  a  worthwhile  effort  to  provide  significant  performance  im¬ 
provement  in  many  cases. 

II.  NCN-LINEAR  ADAPTIVE  DETECTION  PROCEDURES 

One  fundamental  problem  with  the  ncn-Gaussian  signal  processing  is  that 
the  data  is  often  ncnstaticnaxy .  Adaptive  procedures  are  needed  for  estima¬ 
tion  and  detection  of  transient  signals.  Starting  with  the  linear  adaptive 
detection  procedures,  we  have  extensively  explored  the  feasibility  of  Adaptive 
Digital  Filtering  and  Kalman  Adaptive  Filtering  [6]  [7].  As  the  statistical 
characteristics  of  the  data  change  with  time,  filter  parameters  must  be  re¬ 
adjusted  at  periodic  intervals.  This  is  considered  as  a  piecewise  linear 
adaptive  procedure.  If  the  data  statistics  are 

monitored  continuously  and  the  parameters  are  adjusted  with  each  significant 
change  in  data  statistics,  we  will  have  a  nonlinear  adaptive  procedure.  Detec¬ 
tion  of  the  desired  signal  from  the  filtered  output  can  be  performed  by  the 
thresholding  or  correlation  operation. 

The  Adaptive  Digital  Filter  as  shown  in  Figure  1  is  a  ncn-recursive  digital 
noise  by  itself  without  making  Gaussian  assurpticn  of  the  data.  Let  N  be  the 
order  of  the  filter.  The  input-output  relation  of  the  filter  is  given  by 

I^q  bn*»n  f«*-n  C11 

where  is  the  filter  output,  bn  m  is  the  filter  coefficient,  and  fn  is  the 
filter  output.  Define  the  vectors 


SIGNAL  PLUS  NOISE 


Equation  (1)  can  be  written  as 


g_  -  B  '  F  =  F  'B  (2) 

Tn  mm  mm 

By  using  the  Noisy  Least-Mean-Square  Algorithm  [8],  the  filter  coefficients 
can  be  determined  f  ran  the  equations 
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(3) 


where  v  is  a  constant.  After  adjusting  the  parameters  that  include  v,  N,  m, 
and  delay  time,  we  can  obtain  the  desired  result  with  a  greatly  improved  signal- 
to-noise  ratio.  Obviously  the  Adaptive  Digital  Filter  works  well  with  stationary 
data.  Parameter  choice  may  require  a  lot  of  trial-and-error  even  though  the 
filtering  procedure  is  fairly  simple. 

The  signal  model  of  the  Kalman  Adaptive  Filter  is  given  byj 

Vl  ■  Fk  \  *  \  HI 

V*iere  *die  state  variable,  is  the  system  noise,  vj^  is  the  measurement 
noise.  Note  that  Gaussian  assvirptian  is  not  required  of  and  y^,  Given 

a  set  of  measurements  z^,  k  =  1,  2,  — ,  N,  we  would  like  to  estimate  k.  <=  1, 

2,  —  N,  based  on  the  state-space  signal  model  given  fcy  Bq.  C41 .  Since 
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Vi  ■  \+i  *  Vi =  “k  *k  *  wk>  +  Vk+1 


(5) 


Define  an  error  critericn  function# 

Jk+i  *  lzk+i  -  »k  -k +  uk>  -  w2  -  l2 


(6) 


If  \+i  and  ^  are  known,  we  determine  the  values  of  w^,  vk+1  and  f^  iteratively 
such  that  the  state  space  model  is  optimum  in  the  miniumum  mean  square  error  sense. 
The  method  of  Steepest  Descent  can  be  used  in  the  optimization  process  [9].  Once 
the  optimum  values  of  w^,  v^+1,  and  are  obtained,  we  can  perform  a  one-step 
prediction  of  xk+1  by  using  the  equation 


*k+l  =  Fk  \  +  \ 


(7) 


with  w^  being  the  prediction  error.  The  iterative  procedure  for  the  optimiza¬ 
tion  at  the  ith  iteration  is  given  by 

wk,i+l  "  \,i  +  X(2L> 

vk+l,i+l  =  vk+l,i  +  X(2L)  (8) 

Fk,i+1  =  Fk,i  +  X(2Lxk) 

Here  the  increment  A  and  q,  v^+^  q,  and  g  were  given  initially.  In  seismic 
data  processing,  it  is  reasonable  to  assume  that 


Fk,0  =  wk,0  “  Vk+1,0  =  0 

Furthermore,  the  initial  state  can  be  chosen  as  Xq  =  0.  These  assunptions  should 
be  appropriate  for  many  ncn-Gaussian  signal  processing  problems. 

It  will  be  of  interest  to  examine  sate  carputer  results  with  the  seismic 
data  which  is  ncn-Gaussian  [  7  ] .  Both  Adaptive  Digital  Filtering  and  Kalman  Adap- 


tive  Filtering  can  perform  well  to  extract  irrpulse-like  signals  fran  a  corre¬ 
lated  ncn-Gaussian  background  noise.  With  properly  selected  parameters  the 
Adaptive  Digital  Filtering  performs  nearly  the  same  as  the  Kalman  Adaptive 
Filtering.  Figure  2  shows  the  results  of  Kalman  Adaptive  Filtering  for  three 
channels  of  data  with  960  data  points  each.  This  data  segment  represents  a 
section  of  ocrplete  file.  The  upper  part  of  each  photo  is  the  original  data 
while  the  lower  part  is  the  filtered  result.  The  extracted  impulses  occur  at 
nearly  the  same  time  instants.  The  only  parameters  to  be  selected  cue  AL 
(the  increment)  and  ITR  (the  nunber  of  iteration) .  An  empirical  relationship 
between  AL  and  ITR  is  established  for  each  channel  as  shewn  in  Figure  3.  Each 
point  cm  the  curves  represents  an  optimal  oanbination  of  AL  and  ITR.  Although, 
the  curves  are  somewhat  depending  on  the  data  especially  on  the  background  noise 
level,  they  serve  as  a  useful  guideline  for  the  Kalman  Adaptive  Filtering  which 
is  almost  nonparametric  and  is  more  robust  than  the  Adaptive  Digital  Filtering. 
If  automatic  detection  is  desired,  a  correlation  or  a  thresholding  operation 
can  be  used.  Theoretical  detection  performance,  however,  is  not  available. 


As  the  data  statistics  are  general lly  nonstationary,  the  parameters  of  the 
Adaptive  Digital  Filtering  must  be  adjusted  periodically.  No  such  requirement 
is  needed  in  Kalman  Adaptive  Filtering  as  the  final  estimate  for  the  previous 
data  point  can  be  used  as  initial  value  for  the  current  estimate.  Another 
method  to  incorporate  the  time-varying  statistics  in  the  Kalman  filtering  is  to 
use  an  "on-line"  monitoring  process  to  detect  the  significant  change  or  "junp" 
in  the  state  and  update  the  state  vector  accordingly  [10].  This  nonlinear  method 
however  is  limited  to  Gaussian  statistics.  Based  an  the  above  discussion  the 
Kalman  Adaptive  Filtering  method  appears  to  be  most  suitable  for  non-Gaussian 
signal  processing.  Experimental  study  has  demonstrated  good  detection  perform¬ 
ance  with  the  method. 


Fig.  2a 

Channel  1  and  adaptive 
Kalman  filtered  data. 
Ab=0. 0038 
ITR=10 


Fig.  2b 

Channel  2  and  adaptive 
Kalman  filtered  data. 
AL=0. 0028 
ITR=10 


Fig.  2c 

Channel  3  and  adaptive 
Kalman  filtered  data. 
AL=0. 007 
ITR=10 
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III.  Nonlinear  Spectral  Analysis  Techniques 

Both  the  maximum  entropy  method  (MEM)  due  to  Burg  and  the  maximum 
likelihood  method  (MIM)  for  power  spectral  analysis  are  considered  as  non¬ 
linear  high-resolution  spectral  analysis  techniques  suitable  for  processing 
transient  signals.  They  both  bear  a  very  close  relationship  to  ncn-linear 
adaptive  array  processing  techniques  [11].  Furthermore  the  two  methods  are 
very  closely  related  [12].  Experimental  results  have  demonstrated  that 
is  superior  to  MTM  in  spectral  resolution  (see  e.g.  [13]].  Improvement  over 
the  existing  Burg's  MEM  is  possible  by  further  reducing  the  final  prediction 
error  with  the  Fletcher  and  Powell  optimization  method  as  proposed  by  Fougere 
[  14] .  Other  algorithms  that  provide  more  accurate  spectral  analysis  than  the 
Burg's  method  have  also  been  proposed  (e.g.  [15]).  The  maximum  entropy  method 
based  on  the  Fougere ’s  technique  has  been  successfully  implemented  at  the  PDP 
11/45  minioarputer  here  at  the  Southeastern  Massachusetts  University. 


Given  n  data  points  x-^,  x^,  - ,  x^,  we  define  an  (m  +  1)  -  point  pre¬ 
diction  error  filter  (1,  g^,  g^,  - ,  g^)  such  that  the  kth  prediction 

errors  are 


e 


lk 


m 


l 

i=0 


^k+m-i  ^mi 


m 

e2k  l=0  *k+l  ^mi 


k  —  1,2,3,  — ,  i-m 


(9) 


where  g^  =  1  and  and  e^  316  the  forward  and  backward  prediction  errors, 
respectively.  The  mean  square  prediction  error,  or  mean  error  power,  in  both 
time  directions  is 

2  n-m 

P  =°.5(n-m)  l  l  e 

4,1  _  i  .1  i 


(10) 


If  the  prediction  error  filter  of  all  orders  1,  2,  — ,  m  are  gathered 


in  one  matrix  Gm  (with  the  leading  "1"  suppressed) ,  we  may  write 


gll 

g21  g22 

l 

\ 

%1  gm2  "  Srm 


(11) 


Hie  off-diagonal  elements  of  Gm  can  be  determined  fran  the  diagonal 
elements  by  using  the  Levinson  recursion, 


gjk  gj-l,k  +gjj  gj-l,j-k  (12) 

Hie  magnitude  of  t±e  diagonal  elements  of  G^  must  be  less  than  1  in  order 
that  the  prediction  error  filter  be  minimum  phase.  In  order  to  enforce 
this  condition,  we  can  set 

g . .  =  U  sin  e .  (13) 

31  3 

where  is  any  real  angle  and  U  is  a  positive  constant  slightly  less  than 
unity  depending  on  the  computer  used.  For  the  miniocnputer  a  good  choice 
is  U  ■  0.99999.  Once  the  computation  of  the  Burg's  method  is  ocnpleted 
the  gradient  of  resulting  Pm  with  respect  to  0  is  determined  so  that  the 
error  minimization  procedure  can  be  started.  After  a  mmber  of  iterations 
the  filter  coefficients  can  be  recomputed  from  which  the  power  spectrum  is 
determined.  Although  the  original  software  developed  by  Fougere  is  not 
suitable  for  the  miniconputer,  proper  modification  has  made  it  possible  to 


use  this  powerful  technique  in  the  minicaiputer.  Figure  4a  shows  cne 
cycle  of  sinewave  of  1  Hz  at  sanpling  rate  of  20  samples  per  second  . 

Figure  4b  shows  the  Burg’s  spectrum  for  the  20  data  points.  There  is 
spectrum  splitting  and  the  spectral  peak  is  shifted  to  0.92Hz.  The 
Fougere  technique  after  10  iterations  provides  a  correctly  located  peak 
which  is  53  times  of  the  peak  in  the  Burg's  spectrm  (The  vertical  scales 
are  not  the  same  in  the  two  photos.  Every  division  in  the  horizontal 
scale  is  0.5Hz) . 

It  is  important  to  note  that  very  high  resolution  spectrum  can  be 
obtained  from  the  miniccrputer  using  the  method  described  above  at  only 
slightly  increased  computational  complexity  over  the  Burg’s  method.  No 
Gaussian  assumption  is  made  in  the  method.  As  the  method  is  designed  far 
a  small  nunber  of  samples  it  is  particularly  suitable  for  analyzing  the 
spectrum  of  the  transient  signal. 

With  improved  oonputer  hardware  the  method  may  become  useful  for  near  real¬ 
time  operations.  Each  spectral  peak  can  have  seme  physical  meaning  such  as 
in  target  detection  in  clutter.  With  proper  trade-off  in  speed  and  accuracy 
the  method  can  prove  to  be  very  useful  in  non-Gaussian  signal  processing. 

Concluding  Remarks 

Although  only  two  problem  areas  are  considered  in  this  report,  non- 
Gaussian  signal  processing  is  a  challenging  area  which  requires  using  know¬ 
ledge  in  different  disciplines  to  derive  effective  solutions.  This  area 
also  provides  an  opportunity  to  integrate  or  correlate  many  current  signal 
processing  research  activities  which  are  developed  independently.  Continued 
and  strong  research  effort  is  needed  to  pursue  new  directions  and  techniques 
to  solve  seme  difficult  analytical  problems  in  non-Gaussian  signal  processing. 


One  cycle  of  smewave 
at  1  Hz. 


Fig.  4b 

Burg's  Maximum  Entropy 
Power  Spectrum. 
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Fig.  4c 

Fougens’s  Maximum  Entropy 
Pcwar  Spectrum. 
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